Search for: All records

We study the problem of out-of-distribution (OOD) detection, that is, detecting whether a machine learning (ML) model's output can be trusted at inference time. While a number of tests for OOD detection have been proposed in prior work, a formal framework for studying this problem is lacking. We propose a definition for the notion of OOD that includes both the input distribution and the ML model, which provides insights for the construction of powerful tests for OOD detection. We also propose a multiple hypothesis testing inspired procedure to systematically combine any number of different statistics from the ML model using conformal p-values. We further provide strong guarantees on the probability of incorrectly classifying an in-distribution sample as OOD. In our experiments, we find that threshold-based tests proposed in prior work perform well in specific settings, but not uniformly well across different OOD instances. In contrast, our proposed method that combines multiple statistics performs uniformly well across different datasets and neural networks architectures. 

Abstract As cosmic microwave background (CMB) photons traverse the universe, anisotropies can be induced via Thomson scattering (proportional to the electron density; optical depth) and inverse Compton scattering (proportional to the electron pressure; thermal Sunyaev–Zel’dovich effect). Measurements of anisotropy in optical depthτand Comptonyparameters are imprinted by the galaxies and galaxy clusters and are thus sensitive to the thermodynamic properties of the circumgalactic medium and intergalactic medium. We use an analytic halo model to predict the power spectrum of the optical depth (ττ), the cross-correlation between the optical depth and the Comptonyparameter (τy), and the cross-correlation between the optical depth and galaxy clustering (τg), and compare this model to cosmological simulations. We constrain the optical depths of halos atz≲ 3 using a technique originally devised to constrain patchy reionization at a higher redshift range. The forecasted signal-to-noise ratio is 2.6, 8.5, and 13, respectively, for a CMB-S4-like experiment and a Vera C. Rubin Observatory–like optical survey. We show that a joint analysis of these probes can constrain the amplitude of the density profiles of halos to 6.5% and the pressure profiles to 13%. These constraints translate to astrophysical parameters, such as the gas mass fraction,f_g, which can be constrained to 5.3% uncertainty atz∼ 0. The cross-correlations presented here are complementary to other CMB and galaxy cross-correlations since they do not require spectroscopic galaxy redshifts and are another example of how such correlations are a powerful probe of the astrophysics of galaxy evolution. 

Dataset accompanying code and paper: AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs We present AircraftVerse, a publicly available aerial vehicle design dataset. AircraftVerse contains 27,714 diverse battery powered aircraft designs that have been evaluated using state-of-the-art physics models that characterize performance metrics such as maximum flight distance and hover-time. This repository contains: A zip file "AircraftVerse.zip", where each design_X contains: design_tree.json: The design tree describes the design topology, choice of propulsion and energy subsystems. The tree also contains continuous parameters such as wing span, wing chord and arm length.design_seq.json: A preorder traversal of the design tree and store this as design_seq.json.design_low_level.json: The most low level representation of the design. This low level representation includes significant repetition that is avoided in the tree representation through the use of symmetry.Geom.stp: CAD design for the Aircraft in composition STP format (ISO 10303 standard).cadfile.stl: CAD design for the Aircraft in stereolithographic STL file,output.json: Summary containing the UAV's performance metrics such as maximum flight distance, maximum hover time, fight distance at maximum speed, maximum current draw, and mass.trims.npy: Contains the [Distance, Flight Time, Pitch, Control Input, Thrust, Lift, Drag, Current, Power] at each evaluated trim state (velocity).pointCloud.npy: Numpy array containing the corresponding point clouds for each design. corpus_dic: The corpus of components (e.g. batteries, propellers) that make up all aircraft designs. It is structured as a dictionary of dictionaries, with the high level components: ['Servo', 'GPS', 'ESC', 'Wing', 'Sensor', 'Propeller', 'Receiver', 'Motor', 'Battery', 'Autopilot'], containing a list of dictionaries corresponding to the component type. E.g. corpus_dic['Battery']['TurnigyGraphene2200mAh3S75C'] contains the detail of this particular battery. Corresponding code for this work is included at https://github.com/SRI-CSL/AircraftVerse.  Acknowledgements: This material is based upon work supported by the United States Air Force and DARPA under Contract No. FA8750-20-C-0002.  Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force and DARPA.   

Abstract We present a detailed overview of the science goals and predictions for the Prime-Cam direct-detection camera–spectrometer being constructed by the CCAT-prime collaboration for dedicated use on the Fred Young Submillimeter Telescope (FYST). The FYST is a wide-field, 6 m aperture submillimeter telescope being built (first light in late 2023) by an international consortium of institutions led by Cornell University and sited at more than 5600 m on Cerro Chajnantor in northern Chile. Prime-Cam is one of two instruments planned for FYST and will provide unprecedented spectroscopic and broadband measurement capabilities to address important astrophysical questions ranging from Big Bang cosmology through reionization and the formation of the first galaxies to star formation within our own Milky Way. Prime-Cam on the FYST will have a mapping speed that is over 10 times greater than existing and near-term facilities for high-redshift science and broadband polarimetric imaging at frequencies above 300 GHz. We describe details of the science program enabled by this system and our preliminary survey strategies. 

Abstract CMB-S4—the next-generation ground-based cosmic microwave background (CMB) experiment—is set to significantly advance the sensitivity of CMB measurements and enhance our understanding of the origin and evolution of the universe. Among the science cases pursued with CMB-S4, the quest for detecting primordial gravitational waves is a central driver of the experimental design. This work details the development of a forecasting framework that includes a power-spectrum-based semianalytic projection tool, targeted explicitly toward optimizing constraints on the tensor-to-scalar ratio, r , in the presence of Galactic foregrounds and gravitational lensing of the CMB. This framework is unique in its direct use of information from the achieved performance of current Stage 2–3 CMB experiments to robustly forecast the science reach of upcoming CMB-polarization endeavors. The methodology allows for rapid iteration over experimental configurations and offers a flexible way to optimize the design of future experiments, given a desired scientific goal. To form a closed-loop process, we couple this semianalytic tool with map-based validation studies, which allow for the injection of additional complexity and verification of our forecasts with several independent analysis methods. We document multiple rounds of forecasts for CMB-S4 using this process and the resulting establishment of the current reference design of the primordial gravitational-wave component of the Stage-4 experiment, optimized to achieve our science goals of detecting primordial gravitational waves for r > 0.003 at greater than 5 σ , or in the absence of a detection, of reaching an upper limit of r < 0.001 at 95% CL.